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Abstract 



In this paper we consider deterministic limits of molecular stochastic systems 
with finite and infinite degrees of freedom. The method to obtain the determinis- 
tic vector field is based on the continuum limit of such microscopic systems which 
has been derived in . With the aid of the theory we finally develop a new ap- 
proach for molecular systems that describe typical enzyme kinetics or other interac- 
tions between molecular machines like genetic elements and smaller 'communicating' 
molecules. In contrast to the literature on enzyme kinetics the resulting determinis- 
tic functional responses are not derived by time-scale arguments on the macroscopic 
level, but are a result of time scaling transition rates on the discrete microscopic 
level. We present several examples of common functional responses found in the 
literature, like Michaclis-Menten and Hill's equation. We finally give examples of 
more complex but typical macro-molecular machinery. 

1 Introduction 

In this second part we study first the deterministic limit of systems involving small 'com- 
municating' molecules and bigger macro-molecules, which in the following we will call 
'molecular machines'. The communicating molecules will be described as concentrations, 
with the justification that we assume large number of molecules in the reaction volume 
and have derived the continuum limit for this part of the system. In contrast to this, the 
macro-molecular machines being considered will typically only occur in finite numbers. 
Moreover this number will not change during the observation of the system. In case 
numbers are small, we expect the system will have more variance in the observables, i.e. 
it will be more 'noisy'. The construction in [TT] did describe such a system in terms of 
a master equation (ME), where for any communicating molecular species with label i, 
1 < i < N, the evolution of species numbers was described by the probability Po.(n,t) 
given by 





(1) 



dt 



cr'SS 



where n = {n\, ...,njv) 6 represents the collection of numbers of these 'commu- 
nicating' molecules of different species, a € 5 is an index for all different discrete states 
of the Markov chain describing the collection of all molecular machines in the system, 
>C* (n) is a collection of difference operators stemming from a birth-death process, and 
KL T {n) is the transpose of a generator of a Markov chain on S. The above system was 
defined in [TT] as a microscopic system with infinite and finite degrees of freedom, or 
short, an IFSS. Note that as we do not include spatial position to any entity, the discrete 
state space £ indeed describes either one or several finitely many molecular machines, 
all equally accessible by the 'communicating' molecules in one compartment. In other 
words it describes any discrete state attached to any of the macro-molecules in one ore 
several copies inside one such 'well-mixed' compartment. The case of several copies of 
identical machines is subsequently described by having zero transition rates between 
analogous states. The respective structuring of £ with this interpretation is discussed 
in detail in section |3j The continuum limit as derived in [11] next transforms the ME 
into a Fokker-Planck equation (FPE) of the form: 

where e is a parameter dependent on the physic scales of the system. The equation ^ 
is obtained by letting to zero the discretisation implicit in the Master Equation and has 
an asymptotic validity. This is called continuum limit and in its regime e is infinitesimal. 
Physically this corresponds to that the dynamics of the finite state Markov chain (finite 
degrees of freedom) is faster than the dynamics of transitions between the communicat- 
ing smaller molecules, as described by the infinite degrees of freedom. 
Upon e ~ 0, by application of an asymptotic expansion we derived a solution of equa- 
tion (§. This was called adiabatic approximation. The resulting leading order term 
constitutes a deterministic dynamics which turns out to be an average over the different 
invariant measures of the Markov chain. We call this dynamics also average dynamics. 
In [TJ] it is shown that the average dynamics is given as a system of ordinary differential 
equations involving the concentrations, and (a convex combination of) invariant mea- 
sures derived by the asymptotic limit of the Markov chain (MC). The average dynamics 
is given by 



dxi(t) 



^ M(J (xi(i),...,a ;A r(t))4 <T) ( a; i(*)>-> :c iv(i)) ; i = l,-,N, (3) 



dt 

where Xi is the concentration associated to n, and fi is an invariant measure for the MC on 
S. Note that we allow fi to be a convex combination of invariant measures. The functions 
are a collection vector fields describing the dynamics associated to the deterministic 
limit of the ME. With the help of this setting we will develop an alternative approach 
to the description of enzyme kinetics, genetic interactions, ion channel dynamics and 
other macro-molecular performances in a typical cell. As already mentioned we assume 
no spatial structure in this approach explicitly. Spatial relationships are encoded in the 
transition and reaction rates of the system, like in the case of mass-action kinetics. Also 
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typical enzyme kinetics are based on mass-action kinetics with an additional time scaling 
argument. In this respect the theory developed in [11 and this second part is completely 
analogous. Nevertheless we will be able to derive compartmental models, and will have 
one example for such a system. An extension to spatially distributed concentrations for 
the communicating molecules appears to be very natural, keeping the discrete nature 
of the bigger in scale macro-molecular machines. We start developing the theory by 
recalling some properties of the continuum limit and the adiabatic approximation. 

2 The continuum limit and the adiabatic approximation 

Let us now recall the setting introduced in [11] to construct the Master Equation. The 
interaction among molecules depends on the scales at which we study the specific system. 
In particular the interactions depend on the number of molecules involved: the size scales 
of the system. When few molecules interact the stochastic nature of the law of Physics 
prevails, but as the concentrations increase deterministic effects begin to emerge. To 
capture this transition it is necessary to build the description of the dynamics in a form 
that makes explicit reference to the scales. 

Definition 2.1. Let us define two sets of scales 

1. size scales 5 = (Si, Sn), Si > 0, 

2. time scale r > 0. 

The size scales 5 describe the level at which the number of particles are counted. The 
time scale r is the "time-step" at the processes (e.g. chemical reactions) take place. We 
shall see that the continuum limit will be the formalisation of the taking r —* and 
n — > oo, 5^0 keeping nS finite. 

The scales 5 are used to define the space where the chemical processes take place: 
Let be the following lattice 

Lj-={n£ = (n 1 S 1 ,...,n N S N ) : n = (m, njv) G N^} (4) 

Remark 2.1. On we shall define functions, now for fixed 5 the value of any function 
u is uniquely determined by the integer vector n therefore whenever 5 is fixed we shall 
omit the 5 dependence and write u(n). 

At a fixed time the state of a reaction is system is clearly defined by the number of 
different type of particles and this corresponds to a point in L?. Species of particles that 
can be in any number form the infinite degrees of freedom (i.d.f.) of the given system. 
In many reaction network, in particular in biological systems, there degrees of freedom 
that cannot be described as points in some lattice L?. Indeed conformational changes in 
molecules and binding/unbinding events are typical example of configurations which are 
discrete and finite in nature. To describe such digrees of freedom that we called finite 
degrees of freedom (f.d.f.) we introduced a finite set of symbols denoted by E. Therefore 
we define 
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Definition 2.2. The state £ of the system is fully specified by m, ...,nm infinite degrees 
of freedom (i.d.f.) and a second variable, the finite degrees of freedom s (f.d.f.). The 
state ( is therefore the composition 

C = (ni<5i, n N 5 N , a) = (nS, a) £hgx S, 

where hg = ^N^, n is an n-tuple of natural numbers and o runs in a finite set X, 
with |S| = g being the number of discrete states. 

Now the time evolution is determined by a stochastic dynamics, this motivates the 
following definition: 

Definition 2.3. Let the tuple (£,R,P) determine a stochastic process by specifying the 
state (, a set of reactions R, and a vector of probabilities P, such that 

(i) C valued in hg x X, 

(ii) the time evolution of the stochastic process is defined via the set of reactions R 
having three different types: 

(a) Processes involving only i.d.f. 's represented by reactions (possibly reversible) 
of the form 

(n,a) -► (n',cr). 

The operator describing these reactions in the master equation is denoted by 
C* R and has the form C* R = £o ® Saa'j where £q is the same operator for each 
discrete state a = 1, Here b aa i = 1 for a = a 1 , and zero otherwise. 

(b) Processes involving only f.d.f. 's represented by reactions (possibly reversible) 
of the form 

(n,<r) -► (n,o-'). 

The operator describing these reactions in the master equation is the transpose 
K T of the Markov chain generator of the process governing the transitions 
among the discrete states a = 1, ...,<?. The Markov chain is finite dimensional 
with a space of stationary states of dimension strictly less than g. 

(c) Processes involving both i.d.f. and f.d.f. represented by reactions (possibly 
reversible ) of the form 

(n,<r) -► (n',cr). 

The operator describing these reactions in the master equation is denoted by 
C* E . This operator is non-trivial only in the discrete states a which affect 
processes involving i.d.f.. 
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the vector of probabilities 



(Hi) each realisation of the process is valued in x S. The state £ at time t is given by 



P(t,n) = (P l (t,n),...,P g (t,n)), with ^ ^ P a (t,n) = 1. (5) 
The time evolution of P is given by the master equation (ME) 

= (C* R + C* E ) o P(t, n) + JC T (n) P(t, n), (6) 

P, C* R , C* E and K7 are sufficiently regular such that has a unique solution for all 
times t > 0. Then the tuple (£, R, P) is called a (microscopic) system with infinite and 
finite degrees of freedom, or short an IFSS (Infinite-Finite State System). 

2.1 Construction of the continuum approximation 

The ME results from the specification of the reactions at a given scales 5, r. Equation 
^ describes the evolution in time of the probability distribution P. To understand the 
technical structure of the continuum limit one needs to make the following preliminary 
observation. A probability being a measure can be a very "non-smooth" object and 
therefore in the limit S — ► 0, r — > the function P is not expected to have in general a 
"smooth" limit. 

To overcame this difficulty the idea (see [T7] ) is to look for a limit in the space of function 
which are dual to the space of probability measures. The dual of the space of measures is 
in fact a more tame object, and in such a space the ME Q has an adjoint formulation, 
for which the limit can be formulated (see |17[ l20l ITT] ) . 

The ME describes the evolution of a probability measure P a (t; n) according to 

^ = A*[6,t]P, (7) 

where A*[5, r] is the infinitesimal generator defined on the scales 5, r by 

A* [5, t] = C [6, t] + K T [5, r] . (8) 
The operator -4*[£, r] is defined on the space 

P a (t; n):^2Yl n) = 1 for al i I . (9) 

Let us now consider a sequence of scales 5 n , r n such that 5 n — > and r n — > as n — » oo. 
For each index n we have an operator A n = A[5 n , r n ] defined on X* = where the 

configuration space can now be denoted by L n = L? . We ask ourselves what would the 
fate of 081) be as n — > oo. 
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We can think of 5 n —* and r n — > as limit at which space and time step become 
continuous and the numbers of particles are sufficiently to be accounted as densities and 
this motivates the name continuum limit. 

The formulation of the continuum limit can be obtained by using the approximation 
scheme introduced by Trotter in [25] , (see also [20] , [T7J ) . As in [11] to each ^4* defined 
on X* we can associate a vector space X n and an adjoint operator A n . The vector space 
is defined by 



u(t; n, a) : L n x E -* K 9 : IMloo = SU P \ u cr(t, n )| < °o> for al t 



Each X n is dual to X* according to the pairing: 



(10) 



(u,P) n = Yl u a (n)P a (n). (11) 

(n,cr)GL n xE 

The adjoint A n is defined by: 

{A n u,P) n = (u,A* n P) n . (12) 

Let us consider 

u(t, n, a) = ^2 P(ti n > n '> a i o~')u(n , a') 

n'.cr' 

then to equation ([7]) we now associate 



A n U, (13) 



defined on each X„. Here 



see [27] for all the details. 



An = lim-(P*-id) (14) 



For index n equation (13) is the standard Kolmogorov and A n is the infinitesimal gen- 
erator of Markov process on L n x S. 

The definition of the continuum limit is based on the choice of a target space where 
the limit is attained. We shall consider as target the space of continuous function 
X = C (R , M 9 ). The topological dual of X is formed by signed measures on ~R N x S: 

X* = {p(x) : (p, u) < oo, neX}, (15) 

where the pairing is defined by 

(p,u) = dx ^p (J (x)n (7 (x). 
According to [25] we define a sequence of projections 
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Definit ion 2.4. Let '• X 1 — ^ X n ^ t/ic operator that maps w £ X £o l^ni^) £ X^ 
defined as 

V n (u)(k) = u(k5 n ) = u(kiSl,...,k N S^). 
The following holds true (see fl 1) ) 
Proposition 2.1. The projections V n satisfy the following properties 

(i) 1 1 Tn 1 1 n *^ 1 5 

f«J linin^oo HPn^Hn = IkHoo /or e?;er?/ u G X. 

Following (|25j) the projectors T^n allow to define in what sense the spaces X n ap- 
proximate X. 

Definition 2.5. A sequence u n 6 X n converges to u £ if 

W'Pniu) — u n \\ n — > as n — > oo. 

We denote this by u n ~ u. 

We now give the definition for the limit, in fact the continuum limit, of a sequence 
of operators An. This definition is inspired by the one presented in [25]. In fact in the 
present case we have to consider that the operators are functions of the scales 5 n and 
r n . Therefore we set 

Definition 2.6. Let A n : X n i— > X n be a sequence of linear operators. We say that 
A. ~. X i — ► X is the continuum limit of A n ( denoted by A n ~ A) if there exists a sequence 
of scales 5 n ,T n such that 

1. 5 n 0, r n -» 0, 

2. the domain of A is 

D(A) = {u <E X : V n (u) G D(A n ), A n (V n (u)) converges}, 

3. and \\V n (A(u)) — A n (V n (u))\\ n — ► as n — > oo. 

Remark 2.2. TTie dependence on the choice of the scales 5 n andr n makes the continuum 
limit non unique. This is very important because with the choice of the scaling we will 
be able to analyse different type of processes. 

Let us now recall two examples of continuum limits that will be used in what follows. 
We consider 

A* = -(B + - id) (16) 
r 

defined on X^ r as follows: 

A*(P)(m) = V - id)P(m) = P(m + 1) " P(m) . 
r r 



7 



The adjoint A denned on X n is 

. / w s 1 . ,s / s w(m — 1) — w(m) 
A(«)(m) = -(E~ - id)u(m) = — - — , 

T T 

the continuum limit computed in [11 is 

A « A = -c— 
ox 

for 5 — > 0,r — > with <5/r = c > 0. .4 is densely defined on the space of continuous 
functions. The operator A has an adjoint on the space of measures p defined by 

A( P ) = ±{cp). 
Now let us consider the matrix operator 

t \ mk^{o,T) —k (0,t) J 

Its adjoint is 

„ _ 1 / —mk + (5,T) mk + (5,T) 

t\ k-{5,r) -k-(S,r) 

under the condition that for 5 — > 0, r — » and e = e(5, r) = r such that 

k + (S,r) ^5k + k-(8,r) _ fc~ 
r e ' r e 

then continuum limit is 

1 / — xA; + x k + 
,C = ~e{ k- -k- 

Remark 2.3. It is important to observe in the study of the ME the limits 5 — > 0, r — ► 
involve two choices that have to be compatible: 

(i) the limit of the ratio 5 p /r for som p > 0, 

(ii) the choice of the e(S,r). 

In the previous example the important assumption is to have 

k + (6,r) = 5k + , k~{8,T) = k-. 
It is sufficient that these conditions are satisfied asymptotical as 5 — > 0, r — > 0. 
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2.2 Average deterministic dynamics 

Generalising |6] we assume that the Markov chain has possibly more than one stationary 
measure 

M K = {/x(x) : /C T (x)/x(x) = 0}. 

From a modelling point-of-view we assume that we either have a weighted average 
of residence times in which the Markov chain resides in equilibrium, implicitly assuming 
an additional 'microscopic noise' not explicitly modelled which triggers a transition from 
one invariant measure to the next with fixed rates. Or we assume to have a population 
of identical machines like enzymes in which a fraction is in one feasible equilibrium, 
another fraction of the population in the next equilibrium etc.. We avoid trivialities by 
assuming mjc = dim(Myc) < g. Any convex combination 

H=^2e m with °m = i 

m=l m=l 

is in M/c (see [I]). Note that M/c correspond also to all possible invariant measures of 
the Markov chain. Each such measure describes the possible asymptotic behaviour of 
the Markov chain which is in general decomposable, i.e. a product of m/c Markov chains. 
We now take one convex combination [i £ Mx, and construct the adiabatic theory for 
the FPE obtaining an asymptotic expansion in e of p. Let us next consider the marginal 
distribution defined by 

/(x, t) = M x > *) with ^H^^fr/ 0}. (18) 

In [TT] we showed that /(x, t) can be expanded in e and made accurate up to order 0(e). 
Then /(x, t) is determined by 

dJ ^T- = E £ ^C(x)(^(x)/(x,t)) + eT(x)[/(x,t)], (19) 

m=l o-eS 

where T is a parabolic operator. In particular 

rM[/] = E^(r(x)/(x, t) ), 

a/3 M 

where 

I>(x) = -^L^(x)(/C T )^.(x)4(x)^(x), 

and ()C T ) D ('x.) is the so called Drazin inverse of /C T (x). Now one can observe that the 
operator T is parabolic leading to the applicability of standard results from probability 
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theory (see [IS [18]). Such results guarantee that a solution /(x, t) of (19) is the prob- 
ability distribution of a Markov process. In the next section we recall the link between 
/(x, t) and the time evolution of the concentrations x(i). 



2.3 Description in terms of SDEs and ODEs 



The FPE (19) describes the time evolution (up to order of the O(e)) of the probability 
density / of a Markov process. A standard result in probability theory (see again |151I18]) 
links the FPE to an Ito stochastic differential equation (SDE) which gives the trajectory, 
i.e. the realisations of the Markov process. One can show that in our case the stochastic 
differential equation defined on a finite time interval is 

dx a {t) = A a {x(t)) dt + yfe^ cr Q/9 (x(i)) diuf , with a = 1, N, (20) 

P 

and wt being an iV-dimensional Wiener process. It holds that the noise strength 
satisfies [|<r(e, x)|| ~ y/e, and A(x) is the averaged vector field given by 



A Q (x) = ^0 m ^(x)/4 m )(x). 

o-es 



(21) 



Here -C/ CT (x) is the deterministic vector field associated to the finite state j and A{x) is the 
average over the invariant measure //(x) of all vector fields associated to the finite states 
in S. The limit e — > taken in (20) leads to the set of ordinary differential equations 
(ODE) 



dx a (t) 
dt 



A a (x(t)), with q = 1, 



(22) 



We shall call (21) and (22) the average dynamics. If mjc > 1, then the Markov chain 



is equivalent to a product of mx Markov chains and the vector-field (21) describes the 



deterministic dynamics averaged over all rrifc components of E. We illustrate the theory 



using equation (22) and derive as applications effective reaction rates related to different 



macro-molecular machinery. Prominent examples will be enzyme kinetics like Michaelis- 
Menten or more general, Hill's type kinetics. In a forthcoming paper we apply this 
theory to derive rigourously the nonlinear macroscopic model used in [13J to study - on 
a more heuristic basis - the bistability of the Lac-Operon switch inside this framework. 



3 Explicit construction of the discrete state space E and 
the average dynamics 

In this section we analyse the construction of the space £ and the consequences on 
the average dynamics. This will help in the understanding of the theory in relation to 
applications, like the derivation of enzyme kinetics. 
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3.1 The space E 



In modelling it is crucial to construct the discrete state space E of the Markov chain 
in a meaningful and consistent way. For example the modelling step will involve the 
identication of different types of interacting molecular machines present in a process., 
Each such machine will have its own set of different states, its finite degrees of freedom 
forming a subset of E. We therefore give an explicit construction of S in terms of 
compositions of subspaces structuring E. Let us consider a system formed by a certain 
number of chemical species n = (m, ...,ri/v") that can take any integer value, i.e. these 
small molecules can be present in any number in the system. This means n G N , n 
is an i.d.f. and will be treated as a birth-death process, see [TT]. Furthermore let us 
suppose there are now M < oo macro-molecules (eri, &m), each of which can take 
only a finite number of conformations, i.e. forming subset of E. It is useful to introduce 
the notation 

cTjj G Ej, with i = 1, M, 

where Ej = {an, <Ji gi \- Each Ej is the finite set of all possible states for o"y with 
|Ej| = g%. We have two possible ways to construct the total space S out of these sub- 
spaces: 

Definition 3.1 (Product Space). The f.d.f. space consisting of all possible conforma- 
tions (<7i, ...,<tm) is given by the Cartesian product 



where each Ej is finite and therefore |E| = TlfLigi = g. 

Definition 3.2 (Direct Sum Space). The f.d.f. space consisting of all possible confor- 
mations (o~x, ...,o~m) i> s given by the Cartesian product 

where each Si is finite and therefore |E| = Yli=i 9i = 9- 



In the construction of the ME only the transition rates in E enter, obviously affecting 
K, T . It is therefore very useful to chose an enumeration for the elements of E, i.e. we 
can write 

E = {0 1 ,...,O g }, 

where Oj = (ci^, ...,aMi M ), 1 < i < 9- We give a simple example. Suppose that we 
have Ei and E2 given by 

Si = {0-11,0-12}, E 2 = {0-21,0-22,0-23}, 
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then the product space £ = Si x £2 is formed by the following 6 couples: 



£ = Si X £ 2 = {(Cll,<72l), (crii,(722), (fll, CT23), ((712, CT2l), (0"l2,C r 22), (ci2, 0"23)}- 

The direct sum S" = S\ © S2 is formed by 



3.2 Transition rates and infinitesimal generators 

As we have seen the modelling step leads naturally to the idea of composing state spaces 
of Markov chains. The corresponding infinitesimal generators related to each subspace 
will form equivalently the total infinitesimal generator of the system. We discuss this 
construction again for the direct sum and Cartesian product of sub-spaces. It is sufficient 
to consider the case of two such sub-spaces, the construction can then be iterated. 

Definition 3.3. Consider two Markov chains with state spaces £1, £2; and infinitesimal 
generators K,\ and IC2, respectively. Let 



be the transition rates between states a a ^ and a a j both associated to £ a . Then the 
transition rates in the direct sum chain £ = £1 © £2 are denoted by 



With the help of this notation we can make a definition on the direct sum of infinitesimal 
generators. 

Definition 3.4. For two given Markov chains with state spaces £1, £2 and infinitesimal 
generators K,\ and IC2, respectively, the infinitesimal generator K, of the direct sum chain 
£ = £1 © £2 is defined by 



£ = £1 © £ 2 = {(Til, 012, °~21, C22, cr 2 3}- 




o- a ,j, for a = 1,2, 




where 





For the product case the structure of K is more complicated. In fact we have 
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Definition 3.5. Consider two Markov chains with state spaces Si, S2 an d infinitesimal 
generators K,\ and K2, respectively. Let 

&a , ij 

cr a ,i o- a j, for a = 1,2, 

fact ,ji 

be the transition rates between states a a ^i and a a ,j both associated to S a . Then the 
transitions in the product chain S = Si x S 2 = {{o~n, a 2j)}i,j are denoted by 

k ij;i'j> 



{0~li,0-2j) s {o~W, 0- 2 j>), 



k 1 



where 



kij-i'j' 



kl,U> if j = f, 
h,jj< ifi = i\ 
if i ^ i' and j ^ j' 



(23) 



Remark 3.1. The definition (23) is based on the fact that in most of physical examples 
the double transition 



{ii') — > (jf) with i 7^ i! and j ^ j' 



can be neglected. 



Definition 3.6. Let there be two Markov chains with state spaces Si, S2 respectively, 
and infinitesimal generators K.\ and K-2- Then the infinitesimal generator K, of S = 
Si x S2 is defined by 



K (li,2j),(W,2f) 



7' 



where 



Ei'j' = is such that (li, 2j) ^'•n' (li',2f)}. 

The choice of either direct sum or product spaces as the collection of all states of M 
distinct macro-molecular machines is therefore crucially dependent on the interpreta- 
tion of the system under consideration. The product space must be used to model a 
situation where discrete states of different machines are coupled and cannot be attained 
independently, whereas the direct sum models assume complete independence of all the 
states associated to different machines. The notion employed to discuss the asymptotic 
behaviour of a Markov chain is ergodicity. Without going into the details a Markov chain 
is ergodic if each of its states during time evolution is visited again with probility 1 and 
the corresponding attractor is not periodic. Ergodicity is equivalent to the existence of 
a unique invariant measure (see [T]). This is also equivalent to say that the infinitesimal 
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generator K has a unique left-eigenvector (see |19|). The construction of sum and prod- 
uct state space structures with ergodic Markov chains defined on the sub-spaces leads to 
either ergodic or non-ergodic Markov chains defined on E, revealing the nature of each 
type of composition. 

Proposition 3.1. Assume a collection of M finite Markov chains with state spaces 
{Tii}fL 1; where each chain is ergodic. Then the direct sum Markov chain is not ergodic. 

Proof. The direct sum of a Markov chain E has an infinitesimal generator K, which 
is the direct sum of the generators /Q, % = 1,...,M. Each chain JCi has a unique 
invariant measure. This implies that tC has M invariant measures and therefore E is not 
ergodic. □ 

We now consider a simple two-state setting to show that in contrast to direct sum state 
spaces the product of ergodic Markov chains is always ergodic. This setting will be 
generalised in a forthcoming paper based on network theory. 

Proposition 3.2. Assume a collection of M two-state Markov chains with state spaces 
{EjI^Zj and generators 

f ~k% ki 

\ hi -hi 

If for any given Oi,Oj 6 E there exists a sequence of rates Ka t , Ki n j different from 
zero that allow the trasition from Oi to Oj, then the product Markov chain is ergodic. 

Proof. To illustrate the situation one can consider the state space E = {0\, 2 m) 
with 

fel k 2 k 2 M -2 k 2 M -\ 

Ox t=* 2 ^ ... . O 

hi h,2 h 2 M_ 2 h 2 M _ 1 

This implies that the infinitesimal generator associated to the product Markov chain is 
such that 

fCij > 0, for j = % ± 1, and Ku < 0. 

Let Cj = (ICij)f^ 1 be the jth column vector of /C. If for each row i there is j such 
that fCij > 0, then there is no columns has zero entries. In fact all the diagonal entries 
are different from zero in K. Now note that 

M M 

therefore the columns are linearly dependent and det(iT) = and thus rank(IC) < M. 
Now we show that rank{fC) > M — 1 . Assume that there exists A G M. such that at least 
two columns are linearly independent: 

Q = X c m . 
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In particular this implies 
Ku = A/Q m , K. q i = XlCgm with q,m / / and different from the possible row of zeros. 
Now using the expression for fCu we obtain: 

— ^2 K-ip = (A + 1)/Q m and fC q i = \ K, qm . 

This leads to a contradiction, in fact the parameter A cannot be determined since /Qj > 
for some This implies rank(IC) = M — 1 and therefore the equation 

K7 fi = [iJC = 

has a unique solution. 

□ 

3.3 The average dynamics and the choice of S 

The average dynamics is constructed out of two essential data: 

1. A Markov chain with state space S, possibly structured in finite numbers of molec- 
ular machines as discussed before. 

2. A collection of vector fields A^\x) with a G S, the different species of small 
molecules that will be described by concentrations. 

From S one can compute a stationary measure /x(x) and the average dynamics reads 

x(t) = £Mx)AW(x). (24) 

As just discussed in many examples the total space 5 often results from a combination 
of many elementary spaces. We like to clarify the consequences of taking the total space 
as a product or as a sum of the elementary discrete states in more detail. Consider a 
system depending on two 2-state MCs, 

Si = {<Ti, cr 2 }, S 2 = {0-3,(74}. 

In order to be able to derive the average dynamics we need to specify vector fields 
A( CT )(x). The crucial point is that ^^(x) may depend in many ways on the state s, but 
typically there are only two distinct options: 

(i) A^\-x) depends on each single {01, 02, 03, 0-4} 

(ii) A^(x) depends on each couple (<7j, Oj) with i / j. 
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We examine both possibilities. The two available total spaces are 

S sum = Si ©£2 = {01, 02, O3, (J4} 

and 

Sprod = Si X S 2 = {(cri,cr 3 ), ( (7i, (T4), (0-2,0-4), (0-2,0-3)}. 

First one can note that by accident |£ proc z| = |S sum |. Each factor chain Sj has the rates 

01 t — o- 2 , 0-3 — 0-4. 

hi h-2 

Using above rates one can construct the infinitesimal generator associated to S sum . An 
easy calculation gives 



/ -fci fci \ 

hi -hi 

-k 2 k 2 

\ h 2 -h 2 J 

For Tjprod we first enumerate the states 



Oi = (0-1,0-3), 2 = (0-1,0-4), O3 = (0-2,0-3), 04 = (0-2,0-4) 
and construct the rates 

ko k-\ k-\ 

o 1 ^o 2 ^o A ^o 3 ^ Oi, 

fol h2 hi 

which allow us to write the associated generator 



K 



prod 



\ 



ki — k 2 


k 2 


ki 





h 2 


-h 2 - ki 





h 


hi 





-k 2 - hi 


k 2 





hi 


h 2 


-h 2 - 



Now hCsum is not ergodic. In fact the equation JCj um fi = has two solutions 

hi ki 



and 



r'sum 



u {2) = 

r'sum 



ki + hi ' ki + hi 



.0,0 , 



0,0, 



to 



fc 2 



k 2 + h 2 k 2 + h 2 J ' 

The product chain is ergodic. Using /C pro d one finds the unique invariant measure 
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H-prod 



k 2 h x 



kih 2 + hih 2 + k 2 hi + k\k 2 ' k\h 2 + h\h 2 + k 2 h\ + fcifo ' 
A;i/i2 k\k 2 



k\h 2 + /ii/i 2 + k 2 h\ + fcifc 2 fci/i2 + h\h 2 + fc 2 foi + fcifo, 
In the case of the sum space we would obtain an average dynamics of form: 

h\ k\ 



x(t) 
6 2 



/l 2 



fei + /li 

A< 3 )(x) + 



A (1) (x) + 



A>2 



k\ + /ii 
A( 4 )(x) 



A( 2 )(x) 



+ 



k 2 + h 2 ' y " J ' k 2 + h 2 

with 61 + 6*2 = 1. In the case of the product space we would obtain an average dynamics 
of another form: 



x(t) 



/ll/l 2 



k\h 2 + /ii/i 2 + fe 2 /ii + k\k 2 



A( 1 ' 3 >(x)+ 



' /Ci/l 2 + /ll/l 2 + ^2^1 


+ k\k 2 






fclfe + ^1^2 + fc 2 /ll 


+ k\k 2 


fak 2 





A( 1 > 4 )(x)+ 
A( 2 ' 3 )(x) + 
^ 2 ' 4 )(x). 

«1'*2 1" "4 "-2 T K2«l -T K1IC2 

This example nicely shows that the direct sum space assumption leads to fractions 
of work done by the two macro- molecules, something which would become a splitting of 
the population working independently in different modes, based on the assumption there 
are fixed finite numbers of identical macro-molecules working in each such mode. The 
product space creates dependencies among the states of the macro- molecules, creating 
a single mode of operation for the whole population. 

Remark 3.2. Note that under suitable conditions the average dynamics may coincide. 
For example consider the case in which 

A™ (x) = (x) = A« (x) , A^ (x) = A^ (x) = A® (x) . 



■'prod 



reduces to 



Hl A«(x)+ h 



^ 2 )(x), 



The average dynamics with £ 
xff) = 

w ki + hi v ' ' fei + hi 

which is the average dynamics associated to the case X — Yj sum with 9 2 — 0. The reason 
for this is that there exist general relations among the stationary measures associated to 
S sum and S pro rf. One can easily check that for [i S um,i 7^ 
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I^sum,i — / j fJ<prod,(ij) ■ 
j 



4 Applications and examples 

We now present important applications of the theory introduced above. The examples 
will illustrate how the deterministic limit is essentially prescribed by the structure of the 
Markov chain, and the dynamics is affected by new nonlinear terms. Such new terms 
provide the effective dynamics originating from the average procedure. Note that in the 
examples the continuum approximation procedure will be recalled heuristically and in 
particular the continuum approximation of K7 can be computed according to the theory 
developed in jllj . 

4.1 Enzyme kinetics 

This first, at the same time most important example, was already introduced as an 
illustrative example for the continuum and adiabatic limit in |llj . Now we finish it 
with the discussion of the determininistic limit. The Michaelis-Menten and the Hill's 
equation are often used to model reactions that exhibit a saturation behaviour. The 
classical derivation can be found in [12J, and more examples and applications in [5]. 
Among other applications they constitute the basic set of enzyme-catalysed reactions, 
for example in metabolism of the cell (see [E])- We show how an effect described by 
these kinetics can arise as a limit of a multiple-state system, where the switching takes 
place at high frequency. This will be an example of a system with one i.d.f. and one 
f.d.f.. The system has state space (m,Oj) 6 L x £ with £ = {Oq,Oi} and let 5 n ,T n be 
respectively the size and time scales. The f.d.f. Oi is governed by the following set of 
reactions: 

1. with rate k + (5 n ,T n ) jr n and upon the binding of 1 molecule of A the state Oq is 
transformed into the active state 0\ 



2. with rate k (5 n , T n )/T n the state 0\ is decades into the inactive state Oq releasing 
A molecules 



We now prescribe the reactions taking place in each discrete state. In state Oi a certain 
molecule M is produced and degraded according to 



A + O 




Oi -> k ( 5 ™> r ™)/^ O + A. 







V(d; 
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Only in state Oq, the molecules M are degraded according to 

Obviously M is an i.d.f.., i.e. the number of small molecules can reach infinity. 



Remark 4.1. We now assume a >> 1. This implies 

P(t, m, a) = (Po(t, m, a),Pi(t, m,a — a)) ~ (Po(t, m, a),Pi(t, m, a)). 
The master equation for the vector probabilities P{t, m, a) can be written as 

dP 



dt 



ClP + KiP, 



... 



where 

1 / u{S n ,T n ){E+-id)(m-) 
" r n [ o v(6 n ,r n )(E- - id)( • ) + K«5n,r n ) (E+ - id)(m- ) 

with E ± P(m) = P(m ± 1), and 

1 / ak + (5 n ,T n ) -k~(S 



T n \ -ak + {5 n ,T n ) k (5 n ,T n ) 



As in section 2.1 and in [TT] it is shown that the continuum limit is 



c* 



with 



and 



ox ox 



KL 1 



The ME converges to 



a k + —k 
—ak + k~ 



dp i 1 r- 

— = C p + -JCp 
at e 



with e = 0(r n ). 

From this we obtain the two vector fields 



Xq(x) = —ux, X\(x) = — v x + v. 
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The matrix K7 has a unique invariant measure 



1 



ak + + Ar 



k- 
ak+ 



The average dynamics is 



(i„,r(/i(x)/( Q )(x,i))). 



Si ^ 

After some algebra the equation for becomes 



df(°\x,t) 



Of 



vak^ 



k~ +ak + 



(25) 



Equation ( 25 ) becomes 

d/W(x,i) 

at 



d_ 
dx 



-v x + 



u a fc H 



fe- + aAr< 



/ (0) (x,t; 



which is the Liouville equation. This equation is equivalent (see [19]) to the time evolu- 
tion of the concentration x gouverned by the averaged vector field 



X(x) 



k' 



X (x) + 



ak + + k~ ak + + k~ 

Hence the average deterministic dynamics in this case is 



Xi(x) 



dx 
~dl 



X{x) 



-V X + 



ak- 



v. 



(26) 



ak + + kr 

Here we see that the concentration x of M is produced at a rate that depends on the 
concentration a of A, with a saturation behaviour for a large enough. 

Remark 4.2. Note that we can obtain a Michaelis-Menten kinetics. Indeed let us con- 
sider the classical enzyme reaction 

A + E 4^ C -^ hl X + E. 

k-i 

The time scale analysis leads to the Michaelis-Menten rate equation 

dx(t) ^ k 2 e a 
dt a + (fc_i/fci)' 1 ' 

where eo is the steady state for the enzyme concentration [E]. Taking 5 = and 
a = 1 in (26) we recover (21) by setting 

v = k 2 v and k-i/ki = k~ jk . 
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For the two-state system we can also construct the noise. In fact the FPE associated 
to the marginal distribution / is 



and 

f(t,x) = p (t,x) + pi{t,x). 

The FPE is given by 



df(x,t) = d_ 
dx 



dt 



,k+ 



-V X 



ak+ + k- 



v)f(x,t) 



+ 



+e 



ak + k 



d ( d 



(k- +ak+f 

From the FPE one can then derive the associated SDE 



dx(t) 



- ux (t) + ———v-e———{2vx{t)-v) dt 
ak + + k ak + + k 



(2vx{t) - v) 2 + - ( vx(t) 



ak+ 



v I dwt. 



4.2 Formation of macromolecules 

We now analyse the formation of a large macro-molecule like a protein formed at the 
ribosomes, using the mRNS as a matrix. But for simplicity we will not distinguish 
between different types of amino acids. Such a process is often modelled by using a 
generalisation of the Hill's kinetics. The process takes place in several steps, namely each 
new molecule is formed after a sequence of reactions is completed. This sequentiality 
introduces a cooperative behaviour. We assume that the process occurs in g steps. The 
E molecules react with a substrate S in g consecutive reactions, and only after the last 
reaction is terminated a molecule of P is formed. Such a process can be described by 
the following chain of reactions: 



k S ^ k S ^ k S ^ k S ^ 

E ^ — ES ^ — E&2 \ — ESs ^ — 



kS 



ESg P 



We obtain a system with a state space 



E = N x {E, ES, ES 2 , ES g }. 
Let P(p,t) = (Po(p,t), P g (p,t)). The master equation is given by 
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( P (p,t) = -ksP (p,t) + uP 1 (p,t) 

A(P, t) = k S P (p, t) - V Pl(p, t) - k (S - 1) Py{p, t) + U P 2 {p, t) 

P 2 {p,t) = k{s-l)P l {p,t)-uP 2 {p,t)-k{s-2)P 2 {p,t) + uP 3 {p,t) 

{ P g (p,t)=k( S -g + l) Pg-^P, t)~V Pg( P , t) - k p Pg( P , t) + kpPg(p-l,t). 

Let s be the number of particles of type S. Let us assume TO BE CHANGED 

k = ^, v = ^, with e small, 
e e 

For simplicity we consider g = 3. Then this equation can rewritten as follows 

dP _ „ 1 



at 



£* P + -/C J (s) P, 



where 



1 

r 



/ 





V fcp(E- - id )(0 / 



and 







—k s 


V 








\ 


1 




k s 


-u-k(s-l) 









r 







k(s-l) 


-v- k(s - 2) 
















/c s 


— i/ 


/ 



(28) 



K T {s) 



In the continuum approximation we consider s as a parameter. For every s the matrix 
K(s) is a generator of an ergodic Markov chain. Indeed K (s) has generically only one 
zero eigenvalue. The eigenvalues are given by the zeros of 

det(/C - zl) = z A + (3 k(s - 1) + 3 v) z 3 + 

+ (4 kv(s - 1) + 3 k 2 s{s - 2) + 3 v 2 + ku + 2 k 2 ) z 2 + 

+ (A;sz/ 2 + k 2 v s(s-l) + 2k 3 s + k 3 s 2 (s - 3) + u 3 ) z = 0. 

For s > g = 3 there is only one zero eigenvalues and the others are strictly negative. 
The MC is ergodic and its unique invariant measure is equal to 



p(s) 



vl + k u 2 k s + f £:o s ( s _ 1) + k o s ( s ~ ~ 2 ) 



( -I 

UQ 2 k Q s 

u k 2 s(s - 1) 

\ k 3 s(s-l)(s-2) j 
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From the form of C* we construct its continuum approximation C* 
order terms we obtain 



Taking only first 



/ 

















\ 



V -hik ) 

Therefore the deterministic equation for the concentration x p of P is 



dt 



k 3 s(s- l)(s-2) 



kp, 



v 3 + kv 2 s + v k 2 s{s - 1) + k 3 s(s - l)(s - 2) 

where s again is the number of particles of the substrate. The last equation gives the 
precise expected concentration of the product, i.e. formed macro-molecules like a protein, 
where s in this case would be modelling the number of individual amino-acids ready for 
assembly in each step to be attached to the polymer. 
For large s the kinetics reads 



dt 



Hi(s)k p 



k 3 s 3 



k n . 



v 3 + kv 2 s + vk 2 s 2 + k 3 s 3 p 



which is a generalised Hill's term. 



Remark 4.3. Note that in general the number of states g is the maximal exponent in 
the rational function which gives the effective reaction rate. 



4.3 Averaging the average 

This example is meant to illustrate the consequences of having processes which are 
independent but interact through a common background. We have already discussed 
the action of several identical enzymes present in a cell as the typical application. Let 
us consider the following set of reactions 

Ox ^ 2 -^ a 2 + X 
h 

kA 2 

o A o 3 -^ a o 3 + x 

h 

X — >^ 0. 

Here we have in principle two MCs: Si = {Oi, O2} and S2 = {O3, O4}. Let us consider 
the direct sum of them which corresponds to 4-state space 

S = {Oi,0 2 ,0 3 ,0 4 }. 

Each Oi is interacting with a substrate A in two distinguished pools, A\ and A 2 , with 
number of particles a\ and a 2 , respectively. We shall assume that A\ and A 2 remain 
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discrete and therefore don't contribute to the continuum approximation. For brevity we 
skip the construction of the ME and give only the FPE 



where 



/ Ur) 
o 
o 

V o 







d.r 







(t) - £(«■) / 



and 



lC T {x) 



—a±k 


h 







a±k 


-h 














-a 2 k 


h 








a 2 k 


-hj 



We can easily see that in this case dim(M^) = 2. In fact the two stationary measures 
are given by 



/i 



(i) _ 



a\k + h 



( h \ 

a\k 


V J 



(2) _ 



a 2 k + h 



( \ 


h 

\ a 2 k J 



We take the convex combination 



(i = + e 2 ^ 2 \ 

with 9i + 6 2 = 1, and construct the the FPE in the limit e = 0. This is the deterministic 
limit and the FPE becomes the Liouville equation 

^ = (i M ,r(/x/)). 

It turns out that the right hand side is 



aka\ aka 2 
h i — ; — f 2 



f(x,t). 



ka± + h ka 2 + h 
Hence the average dynamics for the concentration of X has the following form 



dx(t) 
dt 



= -2jx(t) + ak 



ai 



a 2 



ka\ + h ka 2 + h 



with 6»i + 6 2 = 1. 



(29) 
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By inspection of (29) one can understand the motivation to call this section "averaging 
the average". In fact, the average dynamics results from averaging over fj,, this is the 
first average. Now /x is a convex combination. One variable (in this case x) is affected 
by the sub-chains Ei,S2, hence the deterministic dynamics contains terms depending 
on both convex parameters. In particular, in this case we have the term 

" ^ +0 2 -^-, (30) 



ka\ + h ka>2 + h ' 

where the convex parameters are 6\ and 62. Due to the normalisation of the convex 



parameters, terms like (30) can be interpreted as averages, namely they describe how a 
variable influenced by the states of different closed sub-chains is affected in the adiabatic 
approximation. 

4.3.1 Many independent molecular machines 

The previous system permits us to make a further generalisation, leading closer to a 
finite population of molecular machines like a larger but finite number of enzymes in 
a reaction volume. Let us assume to have M such molecular mechanisms or machines 
whose dynamics is specified by the following reactions 

Ai + 0-^0+, i = l,...,M, 

hi 

each of which produces X according to the scheme 

of ^ of + x. 



Furthermore we let X degrade, i.e. 



X 



Each machine has a discrete space £$ = {O i , Of} and has a MC with generators /Q 
such that 

— k{ Cti hi 



ki &i hi 



Ki - 

The total space is the direct sum of the Si's, namely the collection 



£ = {0r,0+...,0^,0+}. 

Note that there are no processes linking the state of the i-th machine to the state of the 
j'-th machine. This implies that we have a direct product of MCs whose generator /C 
has a diagonal form 
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K 



( Kx 
K 2 



\ 




\ ... K M j 

One can easily verify that the invariant measure of K is given by 

H = (jui(ai), ...,/ijif(oj|f)), 

where each /ii(cti) is a two-dimensional vector which is an invariant measure for JCf. 
Explicitly we have 



fcj a,- 



A;,- a,- 



kidi + /ij fcj a.j + hi 
Now let us fix a convex combination of measures 

M 



[i(ai, ...a M ) = yjMi(Qi) 



(31) 



where the average dynamics for x will be given by 



dt 



-Sx(t) + y~] r yi9iijLi(ai 



i=l 



Remark 4.4. Let us now consider the normalisation condition on Oi 's. If we set 

M 



(32) 



i=l 



i/jen i/ie kth component of [i is interpreted as the probability that one machine is one of 
the 2M kth- state o/S. It can be useful to consider another normalisation given by 



M 



(33) 



In this case the kth component of /i is interpreted as the number of machines in the kth 
state o/o2x2 MC. 



Let us now fix normalisation ( 33 ) and suppose that all the M machines are equal. This 
implies 

7« = 7, a,i = a, for all 1 < i < M. 
For the average dynamics this implies 



26 



d ^- = -5x{t) + M ll i{a), 

which allows us to conclude that M completely independent molecular machines lead 
to an average dynamics which expression is equivalent to the one of a single machine. 
Only the rate of conversion, as expected, scales with the number of enzymes present in 
the system. 

4.3.2 Different fractions of independent molecular machines 

The preceding examples are useful to construct the following generalisation. Consider 
M molecular machines defined by 

o % ^ o i+2 o l+2 + X, 

hi 

with i = 1 , . . . , M — 2 and degradation 

X 0. 

Using the construction explained in the previous paragraphs one shows that there are 
M Markov chains, with state spaces Sj = (Oj, Oi+2), total space X = ©f£ 1 Sj, and with 
invariant measures 

\ T 'tj rvi tl% T n% J 

From the invariant measures one can easily obtain the average dynamics 

M , 

x(t) = -5x(i) + } 9i at - — 1 \ , 
f-f ki en + hi 



where we use normalisation (33). Now if the M molecular machines are subdivided 
into Q classes of equal machines, then #jS can be interpreted as the fractions of active 
machines. Therefore 

Oi = M h 

and the average dynamics reads 



x{t) = — 5x(t) + Qi a i 



ki Oi 



, ki o-i + hi 

In such a case the average dynamics is equivalent to having Q types of molecular ma- 
chines, each of which contributes with a new rate equal to 

a'i = a.iOi = ai Mi, i = 1, Q. 



27 



4.4 State changes according to a discrete transport model 

We next like to consider a system formed by N compartments or spatial locations, where 
a certain molecule A can pass from one compartment/location to the other. Further we 
assume that from some of these compartments a molecule X is produced or transported 
upon the activation of a molecular machinery. This is a sketch of two typical examples. 
Either the macro-molecule models a simple channel, a membrane protein which opens 
or closes the membrane for a molecule of type X, according to the availability of the 
molecule of type A. Or the system can be interpreted as a sketch of a genetic system, 
where genes are switched on to produce mRNA (type X in this case), according to the 
availability of the transcription factor, in this interpretation type A. One interpretation 
is illustrated in Figure [T] 




<tm 
mo 



open 
A 



"t 



closed 

mo 

m-o 



open 
A 



mo 



mo 



Figure 1: The discrete transport model interpreted to describe single ion channels that can 
be cither open or closed, depending on whether a signalling molecule forms or does not form a 
complex with the channel. The space is discretiscd along the membrane in one dimension. The 
signalling molecules diffuse and therefore either enter or leave any spatial compartment (most 
likely by diffusion from which the transition rates ctj and /3, would have to be computed), and 
this is indicated by the vectors attached to each signalling molecule. 

We denote by Ai the A molecules in the i-th compartment. Consider the following 
chain of reactions 



ai a2. , a:i "jv-i 

^ A 2 ^ A 3 ^ ... A N 

Pi p 3 Pi Pn 



These reactions describe a discrete approximation of A being transported in the N 
compartments. We assume that there is a subset L C [l,...,iV] of compartments at 
which particles can trigger the molecular machinery. Let \L\ = I < N. Let i G L, a 
molecule of Ai can bind to the site (an operator or channel) 0% according to the following 
reactions 
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Remark 4.5. As we have already seen in other examples, there is a Markov Chain with 
generator 

ki&i h{ 



for the two states (O i ,Of). We also have the invariant measure 

kiO>i hi 

Hi 

where as usual [Aj\ = ctj. 



kidi + h ' kiQi + h J ' 



We now assume that a molecule X is produced/transported through the ith machinery 

0% ^ 0% + X. 

To close the system we include that molecules degrade according to 

Ai -^ Sa 0, X 0. 
The operators Oi form a MC whose states are g = 2 l and 

£ = {a = (O h , O k ) with O l G {Or, Of}}. 
Now the processes in which X is produced/transported read 

(O il ,...,0+,...,O il ) ^ (O h ,...,0+,...,O k ) + X, for fc = ii,...,i,. 
For the sake of simplicity let L = {p, q} , we have g = 4 and 

£ = {(0-,0-),(0+ 0-),(0+, 0+),(0-,0+)}, 

with 

(o+,o-)^(o+,o-) + x, 



(0+ 0+) ^ (0 +,o+) + X, 
(0+,0+) -V*(0+ <?+) + *, 

(o-,o+) ->t(o-,o+) + x. 



(34) 



The associated matrix )C T is 
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^ kpQ/p kqdq 



K 1 



hpCLp 
kqdq 



V 

Its invariant measure is 
H{a p , a q ) -- 



hp KqClq 



kqdq 



h q 




hq k p CLp 
k p CLp 





h„ 



h p hq 



-hq - hp J 



hqkpdp 



hp hq I hq kp dp I kqdq kp dp I kq dq hp hp hq I hq kpdp I kqdq kpdp I kqdq hp 



kqdqkpdp 



kqdqhp 



hp hq I ^cj 1 kpdp I kqdq kp dp | I I kqdq kpdp I kqdq hp 

In general if will be more complicated but still ergodic. In fact any state in 5 can 
be reached from any other state. The MC has a unique invariant measure jJL{a£) where 
o-L = (oin-'-Oij). The deterministic dynamics in terms of the concentrations a, and 
[X] = x is given by 



cti(t) = «o + /?202(t) - («i + ^a)oi(^) 

dj(t) = ai_iOi_i(t) + /3 i+ ia i+ i(t) - (a, + A + £4)0* (i), « = 2, 

a N (t) = a N -ia N -i(t) - (Av + c>A)ajv(i) 
x(t) = —Sx x(t) 
x(t) = —5x x(t) + 7 S for s 6 S 



The average dynamics will therefore be 



(35) 



ai(t) = q + (ha,2(t) - (en + <5A)ai(t) 

dj(t) = ai_iai_i(t) + /3 i+ ia i+ i(t) - (a, + A + ^)ai(i), « = 2, iV 

djv(i) = ajv-iajv-i(t) - (Pn + ^4)ajv(t) 
I x(t) = -S x x(t) + ^2 seS ^ s n s (a L ) 
For the simple case L = {p, q} we have: 



(36) 



cti(i) = «o + (ha>2(t) - («i + cu)ai(i) 

cti(i) = aj_iaj_i(f) + /3 i+ ia i+ i(t) - (a, + A + e>AH(i), i = 2, TV - 1 

djv(i) = ajv-iajv-i(t) - (Av + £a)ojv(*) 
»(*) = -fo x(t) + jp(fi 2 (ap(t),a q (t)) + // 3 (a p (i), a g (i)))+ 
, +7 9 (/ 1 iKW. a ?W) + Ma p (t),a q (t))) 



(37) 
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Now note that 



ftp ~r KpClp 

fj,i(a p , a q ) + m{a p , a q ) = — . 



Therefore the equation for x in turn becomes 



x(t)=-s XX (t)+ lp - k : a 1 p{t) u , +i q kgaq{t) 



h p + kpdp{t) h q + k q a q (t) ' 

Remark 4.6. Since the MC is a product of two 2-states MCs, we have that summing 
the invariant measure \x over the possible states of one MC produces the component of 



the invariant measure of the other chain. The form of the reactions (34) implies that 
the contribution of the compartments A p and A q to the dynamics of x are uncoupled. 



Remark 4.7. Consider a modification of reactions \34y into 

(0+,0~)^ (o+,o-)+x, 
(o+,o+) ^ (o+,o+) + x, 

(38) 

(o+,o+) ^(0+,0+) + X, 
(o-,o+) ^(Op,0+) + X, 

where ji ^ 7^ for i ^ k. This would imply that the production/transport of X always 
depends on both states O p and O q . Even though A p and A q are far a part in the chain 
of the compartments their contributions to the dynamics of X are coupled. Indeed one 
easily find that in this case the dynamics of X is 

x(t) = -S x x(t) +7i//i(o p (t),Og(f)) +72/U2(a P (i),a g (t))+ 

+73 A*3 (%>(£)> a <z(*)) + 74A4(op(*)> a q (t)) 
which cannot be reduced to an expression of the form 

x(t) = -S x x(t) + f{a p {t)) + g(a q {t)) 
for some smooth functions f, g. 

5 Discussion 

We have presented a rational and mathematically sound derivation of models describ- 
ing the non-spatial dynamics of large macro-molecular machines finite in number that 
interact with smaller, 'communicating', 'signalling' or 'substrate'- forming molecules in 
the cell. The approach can be used to test various assumptions in one framework, like 
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investigating the effect of small numbers of such molecules on the performance of the 
larger machines, or to recover different types of enzyme kinetics by considering the de- 
terministic limit only. Here the main advantage is that the microscopic assumptions can 
be clearly stated, allowing the framework presented in this series of papers to serve as 
a tool for model construction. The main directions to be discussed further should be a 
more systematic investigation of the noise expected in such molecular systems when the 
smaller molecules vary heavily in numbers. An interesting extension is to analyse the 
problem of large deviations in order to describe situations of bi-stability in the macro- 
scopic equations, again under the influence of noise. This requires to incorporate the 
stochastic fluctuations (e.g. small occupation numbers effect and noise) on long time 
scales. Yet another direction is to consider the adiabatic theory in the context of a 
many-body approach, see for example [2] and |14j . 
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